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Abstract 

We consider the problem of aggregation of incomplete preferences represented by 
arbitrary binary relations or incomplete paired comparison matrices. For a number 
of indirect scoring procedures we examine whether or not they satisfy the axiom 
of self- consistent monotonicity. The class of win-loss combining scoring procedures 
is introduced which contains a majority of known scoring procedures. Two main 
results are established. According to the first one, every win-loss combining scoring 
procedure breaks self-consistent monotonicity. The second result provides a suffi- 
cient condition of satisfying self-consistent monotonicity. 



1 Introduction 



A good method of preference fusion when there are only two alternatives is 
the simple majority of judge votes. This has been shown by Condorcet in his 
famous jury theorem [1]; for a review of further developments see [2,3]. 

The situation becomes much more complicated when the number of alterna- 
tives exceeds two. In this case, the majority of judge preferences may turn out 
to be intransitive and contain preference cycles. Some statistical approaches 
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to this problem are developed in the theory of paired comparisons (see [4]) 
and ranking data (see [5]). 

In this paper, we use a normative approach typical of the social choice the- 
ory and consider indirect scoring procedures as the methods of preference 
fusion. These ingenious procedures are mainly developed in such disciplines 
as applied statistics, scoring of sport tournaments, graph theory, management 
science, etc. Indirect score of an alternative reflects not only the outcomes of 
its comparisons with other alternatives, but also the comparison outcomes of 
those alternatives to which this one has been compared, and this way it may 
depend on all the available preference data. Still there are very few papers on 
indirect scoring procedures in the social choice theory. Probably, the reason 
is that these procedures are not arithmetically ascetic enough, and thereby, 
according to a widespread opinion, do not correspond to the non-quantitative 
nature of the social choice problem. Another problem is that they are not easy 
to describe to the individuals involved in the choice process. However, com- 
plexity may be considered to be an advantage in the context of the strategic 
behavior: complicated procedures are more difficult to manipulate. 

Nevertheless, there is a case where the indirect scoring procedures are really 
needed, namely, the case of incomplete preferences. In this paper, we attempt 
to demonstrate this case, give a brief critical review of indirect scoring proce- 
dures, and compare their properties. 

Let us consider the situation where an individual is given a set of alternatives 
to be compared, but he/she is not an expert in all of them. Suppose she is 
allowed to compare only those alternatives she is familiar with; moreover, to 
report only those comparison outcomes she is certain of. In fact, we consider 
even more general types of individual opinions, namely, each of them is an 
arbitrary binary relation. To prepare these data for calculating indirect scores, 
we represent them by incomplete matrices of paired comparisons borrowed 
from the statistical theory of paired comparisons. 

The main aim of this paper is to examine whether or not the indirect scoring 
procedures satisfy the axiom of Self- Consistent Monotonicity (SCM). It has 
been introduced in [6] where we used it to explore a series of preference aggre- 
gation procedures based on the resolution of discrete optimization problems. 

Up to now there has not been a comprehensive review of indirect scoring 
procedures. However, a lot of information can be found in monographs [4,7- 
10] and papers [11-16]. For relations to the social choice framework see [17]. 
An adjacent subject is the analysis of tournament social choice rules (see, e.g., 
[18,19]). 

The paper is organized as follows. After the following section which provides 
main notation, in Section 3 we give an example demonstrating the specific 
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character of aggregating incomplete preferences. Section 4 contains the state- 
ment and justification of SCM. Section 5 demonstrates that if a procedure is 
based on individual scores, it break SCM. Sections 6, 8 and 10 give a review 
of indirect scoring procedures known from the literature. Section 7 introduces 
the class of win-loss combining scoring procedures and establishes that these 
procedures fail to satisfy SCM. Section 9 provides a sufficient condition for 
SCM, and Section 11 discusses two other axioms. 



2 Notation 

Suppose J = {1, . . . ,n} is a set of alternatives to be compared. There are m 
individuals (judges, voters, etc.), and each of them reports his/her preferences. 
Concerning any pair (i,j) of alternatives, an individual may give one of the 
following four responses: (a) u i is better than j" , (b) 11 j is better than i" , (c) "i 
and j are equivalent", and (d) "I do not report my opinion on this pair." Using 
the responses of the pth individual, an incomplete paired comparison matrix 
= [a?-] is filled out. Its entries a?- and a 1 ^ correspond to the comparison 
outcome of i and j as follows: 

if the pth individual said that 

- i is better than j, then a\j = 1, = 0; 

- i and j are equivalent, then a\j = = 1/2; 

If the individual did not report his/her opinion, a^- and remain undefined. 

By definition, we set all diagonal entries to be zero: a p u = 0, % = l,...,n, 
p — 1, . . . , m. Some authors put = 1/2 or leave a\ undefined, but this does 
not change the results significantly. 

This way we have an array of incomplete paired comparison matrices A = 
(A^\ . . . , A^) which is referred to as a profile of individual preferences. A 
scoring (rating) procedure is a function S from the set of all profiles (or its 
subset) into R n , where s i; the ith component of the resulting column vector 
s = (s-l, . . . , s n ) T is interpreted as a score of the alternative i. All the scoring 
procedures considered in this paper are assumed to be neutral (any reindexing 
of the alternatives preserves their scores) and anonymous (any reindexing of 
the individuals preserves the scores of the alternatives). The term "aggregation 
of incomplete preferences" will mean here nothing but rating the alternatives 
by the scores s l5 . . . , s n \ the greater is a score, the more socially preferred is 
the alternative. 
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3 An Example 



The problem of aggregation of incomplete preferences has a specific flavor 
which cannot be revealed in dealing with complete preferences. To capture it 
consider the following example. 




a. b. c. d. 



Fig. 1. a, b, c: Preferences of nine individuals; d: Combined preferences. 

Suppose there are four alternatives, J = {1,2,3,4}, and nine individuals 
(judges). Three judges are familiar with alternatives 1 and 2 and they all 
feel that 1 is better (Fig. l.a where an arrow from 1 to 2 shows that 1 is 
preferred to 2). Three judges think that 1 and 3 are equivalent (Fig. l.b where 
line segments without arrows designate equivalencies). Finally, three judges 
are familiar with 2 and 4, and they all consider these alternatives of equal 
quality (Fig. l.c). The preferences of these nine individuals are incorporated 
in Fig. l.d. Can we say anything about the comparative quality of 3 and 4? 
Although no judge is familiar with 3 and 4 simultaneously, it is plausible that 
3 is better than 4. Note that 3 and 4 have the same comparison outcomes: 
three equivalencies; they differ only in the alternatives to which they were 
compared. The "opponent" of 3 is probably better than that of 4 and due to 
this 3 may be estimated higher than 4. 

Thus, in aggregating incomplete preferences we should take into account "qual- 
ity (strength) of the opponents" or "caliber of the opposition" or "schedule 
difficulty". These terms came from sport, and we see that there is a similarity 
between the problem of aggregation of incomplete preferences and the prob- 
lem of rating the participants of an incomplete tournament (note that the 
extent of this similarity is a distinct and interesting question). Moreover, in 
the following section this sport analogy will be exploited to explain the mean- 
ing of Self-Consistent Monotonicity, the axiom which is discussed throughout 
the paper. Sometimes the comparison outcome where % is preferred to j will 
be called a "win" of i and a "loss" of j. 
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4 Self-Consistent Monotonicity 



First, let us consider one more example. Suppose we have the incomplete sport 
tournament whose fragment containing all game results of % and j is shown 
in Fig. 2. Suppose we know that a is stronger than b, c is stronger than d, 
and e is stronger than /. Then the results of % are definitely better than those 
of j. Indeed, a loss to a is more pardonable than that to b, a win over c is 
more honorable than that over d, and a win over e is more valuable than a 
draw with /. Besides, i has three extra wins and j has two extra losses, which 
intensify the advantage of i over j. 



In fact, the following self-consistent monotonicity axiom requires that in such 
situations the score of % should be greater than the score of j. The only point is 
the meaning of preconceptions like "we know that a is stronger than b" . Self- 
consistent monotonicity applies to scoring procedures, and here "we know 
that a is stronger than b" signifies "this scoring procedure gives a a greater 
score than 6." This makes clear why this kind of monotonicity is called "self- 
consistent" . 

In the following statement of self-consistent monotonicity, we use the term 
"multiset". Its difference from set is that multiset may contain the same el- 
ement in several copies. This concept is needed here since the outcomes of 
comparisons with different alternatives may coincide. The elements of multi- 
sets will be written within angle brackets. 

Self-consistent monotonicity (SCM). A scoring procedure S is Self- 
Consistently Monotonic if for any set of alternatives J, any profile A, and 
any alternatives i,j<EJ it satisfies the following condition. 

Suppose Ui = (aF ik \ k,p) and Uj = (a% \ £,q) are the multisets of the compari- 



a 




f 



Fig. 2. An illustration to self-consistent monotonicity. 
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son outcomes of alternatives i and j , respectively. Suppose that Ui can be split 
into U- and Uf 1 , and Uj can be split into Uj and Uj 1 in such a way that 

(i) a p ik G U\ implies a p ik = I, a q je G Uj implies a q je = 0; 

(ii) there exists a one-to-one correspondence n of Uf 1 onto Uj 1 such that 
n ( a ik) = a j£ implies a v ik > a q je and s k > s e . 

Then s- > s„-. 

Moreover, if in addition U( is nonempty or Uj is nonempty or at least one 
inequality in (ii) is strict at least once, then s { > s^. 

The application of SCM to a pair of alternatives will be called confronta- 
tion of i and j. 

It turns out that many well-known indirect scoring procedures fail to satisfy 
self-consistent monotonicity. In the following two sections we try to explore 
the features of these procedures that cause them not to satisfy SCM. The 
results are presented in Proposition 1 (Section 5) and Theorem 8 (Section 7). 
Section 9 provides a sufficient condition of SCM (Theorem 12). 



5 Procedures Based on Individual Scores 

A scoring procedure S is based on individual scores if there exist functions 
/ and 5 such that for any profile A = (A^\ . . . , A^) , the corresponding 
score vector s can be expressed as s = 5(sW, . . . , s*" 1 )), where is a partial 
score vector depending solely on the comparison matrix of individual p: = 
f(AM),p=l,...,m. 

The most important instance is 

TO 

P =i 

forming (in case of complete preferences) a class of procedures which satisfy 
the recent axiomatics by Myerson [20]. 

Proposition 1 There are scoring procedures based on individual scores that 
satisfy SCM on complete preferences but no such procedure satisfies SCM for 
incomplete preferences. 

PROOF. To prove the first statement, it suffices to consider the row sum 
procedure, which in case of complete preferences has the form 
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s i =H aP iji i = l,...,n, p = l,...,m, 

3=1 

m 

s = J2 • 

p=l 

This procedure obviously satisfies SCM. 

The second statement can be proved by considering our first example of Fig. 1. 
Let p = 1, 2, 3 be the numbers of individuals in Fig. l.a, p = 4, 5, 6 in Fig. l.b, 
and p = 7, 8, 9 in Fig. I.e. By neutrality and anonymity of S, 



Ji) _ J2) _ ,(3) _ (i) _ (2) _ (3) 

„(4) _ (5) _ (6) _ (7) _ (8) _ (9) 

6 3 — 6 3 — b 3 — 6 4 — 6 4 — 6 4 5 

„(4) _ (5) _ (6) _ (7) _ (8) _ (9) 

64 — S 4 — 64 — 63 — S 3 — S3 • 



Hence, s 3 = s ^ — Z) s 4^ 



Confronting 3 and 4 by SCM (see Fig. l.d), we deduce s 1 — s 2 . But now, 
confronting 1 and 2, we get contradiction. □ 



In words, these procedures generally break SCM because the result of con- 
frontation between two alternatives may be only determined by the preferences 
of the whole board of judges. 



6 Aggregate Scoring Procedures 



Hereafter we will consider scoring procedures that cannot be reduced to in- 
dividual scores. Such indivisible procedures can be called aggregate. A large 
variety of them are representable through the matrix A = [a-] of total com- 
parison outcomes on the pairs of alternatives: 

!0, if a^j is undefined for every p, 
X^ofi, otherwise. (-0 
p 

Consider the scores of the form: 

n 

Si = J2f( a ij' x j)' i = l,-..,ra, (2) 
j'=i 
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where Xj (j = 1, . . . ,n) is some estimate of the alternative j, f is a function 
nondecreasing in both and x y Sometimes averaged scores of the form 

1 n 

Si = — Y,f( a H> x j)> i = l,---,n (3) 

are proposed, where m i is the total number of comparisons of % (see, e.g., [16]). 
For simplicity, we do not consider this modification here, however, it is covered 
by Theorem 8 below. 

6.1 Wei's Procedure 

The most popular form of / in Eqs (2) is the product: 

n 

s i = H a ij x ji i = !,•••,"■ (4) 



Regarding the choice of the form of perhaps the most attractive idea is 
to relate Xj and Sj directly. However, if we set Xj = Sp j = l,...,n, the 
homogeneous system of linear equations (4) may have only a trivial solution 
s 1 — ■ • ■ — s n — 0. A minor modification is to define to be proportional to 

x j = s j /\, j = l,...,n, 
not specifying A a priori. Then 

n 

^ s i = J2 a ij s j> i = l,...,n, (5) 

or in the matrix form, 

As = As, (6) 



thus A is an eigenvalue and s an eigenvector of A. 

This scoring procedure was proposed by Wei [21] and became well-known 
after Kendall's [22] paper. Matrix A generally has several eigenvalues, each 
having its own subspace of eigenvectors. Only one solution is taken, and this 
solution possesses special properties. Let us suppose that the preferences are 
indivisible, i.e., the set of alternatives J cannot be split into two nonempty 
parts J\ and J2 such that for no alternatives j G J2 and i G Ji, > 0. Then, 
by the Perron-Frobenius theorem, the largest in absolute value eigenvalue of 
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A is positive, the corresponding subspace of eigenvectors is one-dimensional, 
and these eigenvectors have the same sign of all terms. Just the normalized 
positive eigenvector s of this type is taken as a vector of scores in the Wei 
procedure (which applies to indivisible preferences). 

Wei's eigenvector can be obtained iteratively. Consider the following sequence 
of score vectors: 



s^il, where 1 = (1, . . . , 1) T ; 
s^is 1 = A 2 l; 



s k = As k-i = A k 1; 



It follows from the Perron-Frobenius theorem that in case of indivisible pro- 
files, the normalized sequence (s fe ) converges to Wei's score vector: 



A k 1 — > s . (7) 



n n - , 

E s k E s k k ^ 
i=i i=i 



It is a simple fact that s k is the number of fc-length paths from % to all ver- 
tices in the preference multidigraph. Thus, in Wei's procedure, alternatives 
are compared by the number of very long ("infinitely-long") paths diverging 
from them in the preference multidigraph. In Subsection 6.3 we shall consider 
a scoring procedure where longer paths are accounted with smaller weights. 

Proposition 2 If indivisible preferences are complete, Wei's scoring proce- 
dure satisfies SCM, but it breaks SCM for incomplete indivisible preferences. 



PROOF. To prove the first statement, confront arbitrary i and j by SCM: 
1 ™ 

s i = -rY, a ik s k, (8) 
A k=i 

l n 

Sj = TY, a jk s k, (9) 

A fc=l 



and assume that some partitions of Ui and Uj described in self-consistent 
monotonicity exist. Then Uf 1 induces on the right-hand side of Eq. (8) a no 
lesser (respectively, greater in the strict case) sum of terms than U 1 , 1 induces on 
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the right-hand side of Eq. (9). The same can be said of U( and Uj . Therefore, 
Si > Sj (sj > Sj in the strict case). 

To prove the second statement, consider the cumulative preference digraph 
depicted in Fig. 3. 















3 / 


5 









Fig. 3. Preferences in the proof of Proposition 2. 

The comparison outcomes of alternative 5 are equivalencies with alternatives 1, 
2, 3 and 4. Alternative 5 is introduced only to make the preferences indivisible. 
Consider the equations with s 2 and s 3 on the left-hand sides: 

As 2 = s 4 +s 5 /2, 
\s 3 = s 4 +s 5 /2. 

Hence, s 2 = s 3 , which contradicts SCM, since 3 has an extra "loss" (a 31 = 
0, a 13 = l). □ 

The example used in the proof of Proposition 2 reveals one important fea- 
ture of Wei's procedure as applied to incomplete preferences. Namely, this 
procedure bases itself only on "wins" and does not take into account "losses" . 
More precisely, it does not distinguish "losses" from missing comparisons (and 
precisely this contradicts SCM). A possible way to make the procedure more 
balanced is to also consider a dual procedure which is based on "losses" , and 
then to combine the resulting scores. This has been done by Hasse [23] and 
Ramanujacharyulu [24]. 

6.2 The Procedures by Hasse and Ramanujacharyulu 

Let w and 1 denote the right and left eigenvectors of matrix A, corresponding 
to its maximal eigenvalue A : 

n 

^Wi = J2 a ij w j> i = l,---,n, (10) 
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n 

Mi = J2 a jitj, i = l,...,n, (11) 



Here w = (u^, . . . , w n ) is the vector of Wei's win-scores, which is not sensitive 
to the distinction between "losses" and missing comparisons; 1 = (£ l , . . . ,£ n ) 
is the vector of loss-scores, which does not distinguish "wins" and missing 
comparisons. The matrix form of Eqs (11) is 

Xl = B\, (12) 
where B = A T is the transpose of A. 

Hasse [23] proposed to combine w and 1 into the ultimate scores: 

s i = w i -£ i , i=l,...,n. (13) 

Another procedure proposed by Ramanujacharyulu [24] has scores given by: 
s i = w i /£ i , i = l,...,n. (14) 

These two procedures are close but may produce different rankings, since the 
first one registers the absolute differences between win-scores and loss-scores, 
whereas the second is based on the relative differences. Both procedures are ap- 
plicable to indivisible preferences and turn out to break self-consistent mono- 
tonicity. We do not formulate the corresponding statement, because there are 
many possible modifications of such procedures, and a more reasonable thing 
to do is to isolate their common features and then to prove a more general 
theorem. To this end, in the following two subsections we consider two other 
families of indirect scoring procedures, and then proceed with a theorem. 

6.3 The Procedure of Katz-Thomp son-Taylor 

As we have seen, Wei's scores rank order the alternatives according to the 
number of very long paths diverging from them in the preference multidi- 
graph. Katz [25] and then Thompson [26] and Taylor [27] proposed a scoring 
procedure in which the long paths play just a correcting role, whereas the 
short paths are taken into account with greater weights. Namely, Thompson 
introduced the scores of the form 

w = (A + eA 2 + e 2 A 3 + •••)!, (15) 
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where e is a small positive parameter, lie < r _1 where r is the spectral radius 
of A, then the series (15), which in fact is a geometric progression, converges 
and 

w = A(I-eA)- 1 !, (16) 



where I is the identity matrix. 

Thompson shows that his normalized vector w tends to Wei's eigenvector as e 
approaches r~ l from below. Besides, he gives a game-theoretical interpretation 
of these scores comparable with the approach by Laffond, Laslier and Lebreton 
[28]. 

The paper by Taylor is one of the few social choice articles devoted to indirect 
scoring procedures. He investigated sequential voting by the committee and 
considered Hamiltonian sequences of winning alternatives, which led him to 
a scoring procedure very close to (15). In fact, Taylor used another matrix 
C = •] instead A, where 



-dji, if a tj -aji > 0, 
1 0, otherwise. 



to, 



Since for the preference structure we use in the proof of Theorem 8 in Section 7, 
A and C coincide, the statement of this theorem is valid for Taylor's procedure. 

As above, w in Eqs (15) and (16) is a vector of win-scores, and it is worth 
being supplemented by the corresponding loss-scores. So we may consider a 
pair of vectors (w, l) where 

1 = (B + eB 2 +e 2 B 3 + •••)!, (17) 



and combine w and 1 as in the procedures of Hasse and Ramanujacharyulu: 

s i = w i -£ i , i = l,...,n 

or 

s i = w i/^ i = l,...,n. 

The question we are interested in is whether or not these procedures satisfy 
self-consistent monotonicity. The answer is negative. After considering one 
more family of scoring procedures, we give a theorem that will clarify this 
point to some extent. 
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6.4 Directed Tree Procedure by Daniels-Ushakov-Goddard-Levchenkov 
Let c[ and cj be the total "win" and total "loss" of i, respectively: 

n 

C i = a ij 5 
n 

c i=H a jn i = !,•••,"■ (18) 

Daniels [11] and other writers [29-33] proposed, with various motivations, the 
scores w ± , . . . , w n that satisfy 

1 n 

Wi = —Y, a ij w ji i = l, (19) 

The only difference of this system of equation from that of Wei is that A is 
replaced by . This is possible because the matrix [a, - /cj] always has a unit 
eigenvalue. Scores iu i have several interesting interpretations listed below. 

(i) u>j is proportional to the number of directed trees diverging from i in the 
preference multidigraph [34]; 

(ii) w i are proportional to the final probabilities of the "winning Markov chain" 
[29,30,35,32,33,36]; 

(iii) w is the vector of "fair bets" in the model of Moon and Pullman [12]. 

Also, in [37-39] this procedure is analyzed in the social choice framework, and 
in [40,41] its strategic properties are touched upon. 

Exactly as before, we may consider the corresponding loss-scores mentioned 
by Daley [35]: 

1 n 

li = -Y, a a^ * = !,•••, ™ (20) 

Ci j=1 

and define the ultimate score vector as a combination of w and 1, e.g., 

s i = w i -£ i , i = l,...,n 

or 

s i = w i/^ i = l,...,n. 
The procedures of this family break SCM, and now we are in a position to 
specify a more general class of indirect scoring procedures that do not obey 
SCM. All the above procedures are included. 
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7 A General Class of Procedures That Fail to Satisfy Self-Consis- 
tent Monotonicity 



In this section we define win-loss indices and win-loss combining procedures 
and prove that such procedures break self-consistent monotonicity. 

In the following Definitions 3-5, profile A is fixed; "vector" means n- 
component column vector. 

Definition 3 (recursive) . Win-loss index of finite order. 

1°. A pair of vectors (w, l) is a win-loss index of zero order if w = \ = 
(1 I)'- 

2°. A pair of vectors (w, l) is a win-loss index of order k if there exists a win-loss 
index (w', l') of order k — 1 and functions f , g, a and (3 such that 

«>i= ,1 -J (WD, 

li= a. 1 r+ ^ (L'i), < = l,...,n, (21) 

P\ C i J C i ) 

where Wl = ((a n , w\), (a i2 , w' 2 ), . . . , (a in , w' n )), L- = ((a H , fj, (a 2i , £' 2 ), 
. . . , (a ni , £' n )), i = 1, . . • , n; angle brackets designate multisets. 

W[ and L[ are respectively called the win-performance and the loss-perfor- 
mance of i corresponding to (w',l'). 

One can see that in the Directed tree procedure of Subsection 6.4, a(c+, cl) is 
c~l\ P(ci , c+) is c\\ f and g are sums of products. An essential difference is that 
in the Directed tree procedure, w i and l i are related with the win-performance 
and loss-performance corresponding to the same pair of vectors (w,l), not to 
a win-loss index of the previous order. Thus, Eqs (19) and (20) do not fit 
Definition 3. That pair of vectors (w, l) satisfies the following definition. 

Definition 4 Win-loss index of infinite order. A pair of vectors (w, l) = 
((w 1 , . . . , w n ) T , (£ ± , . . . , i n ) T ) is a win-loss index of infinite order if there exist 
functions f , g, a and {3 such that 



where Wi = ((a a , wj, (a i2 , w 2 ),..., (a in , w n )) and Li = ((a u , ij, (a 2i , £ 2 ),..., 
(o n iiO)> i = l,...,n. 
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The following definition introduces win-loss indices without order. 

Definition 5 Win-loss index. A pair of vectors (w, l) = ((w 1 , . . . , w n ) T , 
(£ ± , . . . ,£ n ) T ) is a win-loss index if there exists a sequence ((w 1 ,^), (w 2 ,£ 2 ), ...) 
of win-loss indices of finite or infinite order and functions ip and ip such that 
fori = l,...,n, 

w t = <p(wl,£l,w 2 ,£ 2 ,...), 

£ t = i,{w),£\,uo 2 ,£ 2 ,..). (23) 

The components of w and 1 are referred to as win-scores and loss-scores, 
respectively (note that they depend on both w-components and ^-components 
of the sequence elements). The next (and last) definition introduces a scoring 
procedure that combines win-scores and loss-scores of a win-loss index. 

Definition 6 Win-loss combining scoring procedure. Scoring procedure S is 
a win- loss combining scoring procedure if for any profile A there exists a win- 
loss index (w, l) and a function h such that 

s i = h{w i ,£ i ), i = l,...,n. 

Proposition 7 All scoring procedures that generate score vectors of Section 6 
or any their iterative approximations constructed as in 2° of Definition 3, are 
win-loss combining scoring procedures. 

The proof of Proposition 7 is straightforward. Among other win-loss combin- 
ing scoring procedures, let us mention the elaborate procedure by David (see 
[42,4,15], and procedures by Cowden [43] and Ginovker [44]. The latter two 
authors also proposed other procedures which will be mentioned below. An 
interesting discussion of related topics can be found in [16]. 

Theorem 8 Every win-loss combining scoring procedure defined on the set of 
indivisible preference profiles breaks self- consistent monotonicity. 

PROOF. Assume that there exists a win-loss combining scoring procedure 
that satisfies SCM on the set of indivisible preference profiles. Suppose that 
(w, l) is its win-loss index for the profile with the preference digraph shown in 
Fig. 4 and that s i = h{w i , i — 1, . . . , n. 

Lemma 9 In the above assumptions, (i) £ 2 = £ 3 ; (ii) w 2 = w 3 . 

Proof of Lemma 9. First, assume that (w, l) is a win-loss index of kth order. 
If k — 0, (i) and (ii) of Lemma 9 hold. Note also that w 6 = w 7 . If k > 0, there 
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Fig. 4. Preferences in the proof of Theorem 8. 

exists a win-loss index (w', 1') of (k — l)th order and functions /, g, a and j3 
such that Eqs (21) hold for % — 1, . . . , 7. In particular, 

W 6 = ^2)/ (((1, O, (0, w> 2 ),..., (0, w' 7 ))) = w 7 . 

(i) Similarly, £ 2 = h- 

(ii) w 2 = (((0, K), . . . , (0, w' 5 ), (1, «/ 6 ), (0, 

» 3 = ^i)/ (((o, K), • • • , (o, K), (l, «/ 7 ), (o, <))). 

In the latter formula we interchanged the two last elements of the multiset 
W%. As we have seen above, for every win- loss index of finite order, w 6 = w 7 , 
hence w' 6 = w' 7 . Therefore, w 2 = w 3 . 

For win-loss indices of infinite order, (i) and (ii) are proved similarly. 

Suppose now that (w, l) is an arbitrary win-loss index. Then there exists a 
sequence of win-loss indices of finite or infinite order ((w 1 , i 1 ), (w 2 , £ 2 ), . . .) and 
functions <p and ip such that 



£ t = ij(wl£lw 2 J 2 ,...), i = l,...,7. 

Since for every t = 1,2,..., i\ — £\ and w 2 = w l Z) we get l 2 = £ 3 and w 2 = w 3 . 
Lemma 9 is proved. 
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Since s 2 = /i(u> 2 ,£ 2 ) and s 3 = h(w 3 ,£ 3 ), Lemma 9 implies s 2 = s 3 . 

Let us contrast alternatives 2 and 3 by self-consistent monotonicity. It follows 
that if either s 6 > s 7 or s 7 > s 6 , then s 2 = s 3 is impossible. Hence, s 6 = s 7 . 
Now, contrasting alternatives 4 and 5, we get s 4 > s 5 . 

Finally, contrasting 6 and 7 and using the proved statements s 2 = s 3 and 
s 4 > s 5 we get s 6 > s 7 , which contradicts s 6 = s 7 obtained above. Thus, our 
assumption is wrong, and self-consistent monotonicity is broken. □ 

Theorem 8 can be explained as follows. All functions in Definitions 3-6 are 
arbitrary, we impose no constraints on them. The only non-arbitrariness was 
that "wins" and "losses" were treated separately (whereas "draws" influenced 
both win-scores and loss-scores). This separation was not complete, because 
the expressions (21) and (22) for win-scores w i also contained the total losses 
c7 t and vice versa; such a cross-dependence extended to win-indices and loss- 
indices (23) too. In spite of that, as can be shown, a win of alternative j over 
alternative k can affect the win-score of % ^ k only if there exists a directed 
path in the preference multidigraph from % to j or from % to k. Similarly, this 
win of j over k can affect the loss-score of i ^ j only if there exists a directed 
path from j or from k to %. Therefore the ultimate scores are sensitive only 
to directed paths of these two kinds. As can be concluded from the proof of 
Theorem 8, self-consistent monotonicity implies the sensitivity of the ultimate 
scores to the influences that spread not only through directed paths, but also 
via paths with altering directions of arrows. 



8 Win-Loss Unifying Procedures 

In this section we consider two scoring procedures that do not separate wins 
and losses in the score equations, treating them uniformly. Another important 
difference from the procedures of Section 6 is that here the function combining 
the comparison outcomes and the estimates of the opponents (function / in 
Eqs (2)) has an additive (instead of productive) form. It is worth noting that 
if the score of i is represented through multiplicative terms such as Sj, then 
the influence of Sj upon the score of i depends on a^: the greater a^, the 
stronger this influence. In the extreme case where a,-,- = 0, s„- is not taken into 
account at all. An additive form of scores, on the contrary, causes a uniform 
influence of Sj upon s iy regardless of a^. This appears more reasonable. 
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8.1 The Least-Squares Procedure 



This procedure, first proposed by Smith [45] and Gulliksen [46] and then 
investigated in [47-52, etc.], constructs a mean square approximation of the 
comparison outcomes by the differences between the desired scores: 

minimize ^ (r£- - (s t -s^)) 2 , (24) 



where 

r% = a?- - a% (25) 



is a skew-symmetric modification of the comparison outcomes; rfj are unde- 
fined whenever a^- and are undefined; the sum extends over those i, j and 
p for which r?- is definite. 

Partial differentiation reduces the problem (24) to the system of linear equa- 
tions 

Si = — J2( s j +r ij)' i = 1, • • • , n, (26) 



where = c+ + q is the number of comparisons of i. 

Note some parallelism of this system of equations and that of (19). A multi- 
plicative counterpart of Eqs (26) is considered in [16] and fits (3). 

This procedure is applicable not only to indivisible preference profiles, but to 
all profiles with connected preference multigraph. Under this condition, the 
rank of the system (26) is n — 1, and scores s 1 , . . . , s n can be found up to 
an additive constant. For the complete preferences, this procedure reduces to 
that of row sums, and it is rather well-performing for incomplete numerical 
(weighted) preferences. However, in the case of discrete incomplete preferences 
which we consider here, it can produce some unnatural results (see the proof 
of the following proposition). 

Proposition 10 The Least squares procedure breaks self- consistent monoto- 
nicity. 



PROOF. Consider the preference multidigraph shown in Fig. 5. 

The least-squares score of 2 is greater than that of 1, in spite of that 1 has an 
extra win. This breaks self-consistent monotonicity. □ 
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Fig. 5. Preferences in the proof of Proposition 10. 



This proof can be commented as follows. The Least-squares procedure better 
fits numerical preferences; it punishes 1 for the win over 4 because it "expects" 
that if 1 beats 3 and 3 beats 4 with the same "intensity" , 1 should beat 4 with 
a greater intensity. The following procedure eliminates this flaw. 

8.2 The Generalized Row Sum Procedure 

This procedure [53,54] can be considered as a Bayesian modification of the 
previous one. It is derived axiomatically and has Markov chain and graph 
theoretic interpretations [55,56]. The scores satisfy the system of equations 



where e is a positive parameter, e < (m{n — 2)) 1 , and 7 = mn + e 1 . 

Proposition 11 The Generalized row sum procedure satisfies Self- Consistent 
Monotonicity. 

The proof is given in [6]. The form of these scores suggests what kind of 
procedures satisfies self-consistent monotonicity. A class of such procedures is 
described in the following section. 



9 A Sufficient Condition of Self-Consistent Monotonicity 

Theorem 12 Suppose that a scoring procedure S is such that there exists a 
function f defined on finite nonempty multisets of real triples and possessing 
of the following properties. 




(27) 



19 



(i) for every A, 

f(((c$ j ,s i ,s j )\j,p)) = 0, i = l,...,ra. (28) 

(a) f is 

- increasing in every afj, 

- increasing in every Sj (j ^ i), 

- decreasing in s i . 

(Hi) Let Vi = ((a?-, s i; s^) | j,p). For every .4. and for every i,k e J, 

(0, s„ Sfc ) e Vi, ften /(^ \ ((0, s„ Sjfc )» > /(^)- 
Then scoring procedure S satisfies self- consistent monotonicity. 

PROOF. Assume that the relation between % and j described in the state- 
ment of self-consistent monotonicity holds, but s { < s ■ (respectively, s i < Sj 
in the strict case). Then, by (ii) and (iii) of Theorem 12, the left-hand side 
of the ith equation in the system (28) exceeds the left-hand side of the jth 
equation, and hence they both cannot be equal to zero. This contradiction 
proves the theorem. □ 

In [57] a modification of SCM is proposed which is met by those and only 
those scoring procedures that have the implicit form (i)-(ii) of Theorem 12. 



10 Four Procedures Satisfying SCM 



Table 1 lists four procedures that satisfy self-consistent monotonicity. 
Table 1 

Scoring procedures that satisfy SCM. 



Procedure 


Domain* 


/(•) of Theorem 12 


Zermelo [58], Bradley&Terry [59],... 


ID 




Daniels [11], Ginovker [44] 


ID 


f ( s i s i\ 


Cowden [43] 


ID 


J2(aij 8j(l - - aji s^l - Sj)) 
j,p 


Generalized row sum procedure [54] 


U 


£ E(7^- - (Si-Sj)) - Si 
j,p 



* ID means indivisible preference profiles, U all profiles. 
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The second column conveys the set of preference profiles to which the corre- 
sponding procedure is applicable. Of the four methods, the Generalized row 
sum procedure is based on the solution of a linear system of equations, three 
others are reduced to nonlinear systems of equations, which are usually solved 
with iterative algorithms. 

A question arises of how to compare further the procedures that satisfy self- 
consistent monotonicity. The last section contains two remarks on the possible 
additional axioms. 



11 Concluding Remarks 

In a follow-up paper we will continue the axiomatic testing of indirect scoring 
procedures and turn to their axiomatic derivation. We now give two possible 
conditions which can supplement self-consistent monotonicity. 

11.1 Macrovertex Independence 

The main idea of self-consistent monotonicity is sensitivity of the aggregating 
procedure. However, there exists an independence condition called Macrover- 
tex independence that does not contradict SCM and seems rather natural (cf. 
the discussion after Theorem 8). 

A subset M of the set of alternatives J is called a macrovertex of the preference 
multigraph if for every i,j<EM and every k G J \ M, = n j7c , where n ik is 
the number of comparisons between % and k in A- 

Macrovertex independence. The comparison outcomes of the alternatives 
within a macrovertex have no influence on the scores of the alternatives outside 
the macrovertex. 

11.2 Splitting Balance 

Let us note that self-consistent monotonicity is not a very restrictive condition, 
since it leaves room for some preconception. To conclude with what we started, 
we address our first example again and give it a sport interpretation. Suppose 
that Fig. l.d depicts an incomplete chess tournament. Assume that 4 is the 
world chess champion, 1, 2 and 3 being university students. Our aim is to 
estimate the strength of the players taking into account prior information. 
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Most probably, the world champion is the strongest player, and we should 
allot the highest rank to 4. Player 2 managed to make three draws with the 
champion. That is great, and we rank 2 second in spite of three his/her losses 
to I. Then 1 is the third and 3 the last. Note that this rank order does not 
contradict SCM! This is what we meant by saying that SCM leaves room for 
some preconception. There exists a logic (a bit biased) that justifies ranking 
(4, 2, 1, 3). To avoid this, the following condition may be added. 

Splitting Balance. Suppose the set of alternatives J can be split into J\ and 
J 2 such that for no i G J\ and j G J 2, a-j > 0. Then there exist i G J\ and 
j G J2 such that s i > s-. 

The objective of this paper was to put together various indirect scoring proce- 
dures (mainly, of infinite order) and to subject them to the analysis in the spirit 
of the social choice theory. We considered the case of incomplete preferences 
and an axiom we referred to as self-consistent monotonicity. It was established 
that many of the known indirect scoring procedures are win-loss combining 
procedures and, by Theorem 8 of Section 7, they break self-consistent mono- 
tonicity. In our opinion, some of them are applicable in the analysis of social 
networks, since here self-consistent monotonicity is not so attractive as in 
preference aggregation (cf. [25,60,61]). For instance, the popularity index of 
an individual should not strongly depend on his/her own responses. 

In Theorem 12 we proposed a sufficient condition for self-consistent monoto- 
nicity and then listed four procedures that satisfy it. The analysis of indirect 
scoring procedures started in this paper is intended to be continued with 
additional axioms (two possible conditions were mentioned in this section), 
and with the end goal of axiomatic construction of such procedures. 



References 

[1] M. J. A. Marquis de Condorcet, "Essai sur l'application de Panalyse a la 
probability des decisions rendues a la pluralite des voix", Paris, 1785. 

[2] B. Grofman and G. Owen, "Review essay: Condorcet models, avenues for further 
research", in Information Pooling and Group Decision Making (authors, ed.), 
pp. 93-102, Jai Press, Greenwich, CT, 1986. 

[3] S. Berg, "Evaluation of some weighted majority decision rules under dependent 
voting" , Mathematical Social Sciences, Vol. 28, pp. 71-83, 1994. 

[4] H. A. David, "The Method of Paired Comparisons", Griffin, London, 
Second Edn, 1988. 



22 



[5] M. A. Fligner and J. S. Verducci, eds., "Probability Models and Statistical 
Analyses for Ranking Data", Springer- Verlag, New York, 1993. 

[6] P. Yu. Chebotarev and E. Shamis, "Constructing an objective function 
for aggregating incomplete preferences", in Econometric Decision Models 
(A. Tangian and J. Gruber, eds.), Lecture Notes in Economics and 
Mathematical Systems, pp. 100-124, Springer- Verlag, Berlin, 1997. 

[7] J. W. Moon, "Topics on Tournaments", Holt, Rinehart and Winston, New 
York, 1968. 

[8] A. R. Belkin and M. S. Levin, "Decision Making: Combinatorial Models of 
Information Approximation" , Nauka, Moscow, 1990. (In Russian). 

[9] R. Van Blokland-Vogelesang, "Unfolding and Group Consensus Ranking for 
Individual Preferences", DSWO Press, Leiden, 1991. 

[10] W. D. Cook and M. Kress, "Ordinal Information and Preference Structures: 
Decision Models and Applications", Prentice-Hall, Englewood Cliffs, NJ, 1992. 

[11] H. E. Daniels, "Round-robin tournament scores", Biometrika, Vol. 56, pp. 295- 
299, 1969. 

[12] J. W. Moon and N. J. Pullman, "On generalized tournament matrices", SIAM 
Review, Vol. 12, pp. 384-399, 1970. 

[13] H. A. David, "Ranking the players in a round robin tournament", Review of 
International Statistical Institute, Vol. 39, pp. 137-147, 1971. 

[14] D. S. Shmerling, S. A. Dubrovskii, D. Arzhanova, and A. A. Frenkel', "Expert 
judgments: Methods and applications", in Statistical Methods of Analysis of 
Expert Judgments, Uchen. Zap. Stat., Vol. 29, pp. 290-382, Nauka, Moscow, 
1977. (In Russian). 

[15] H. A. David and D. M. Andrews, "Nonparametric methods of ranking from 
paired comparisons" , in Probability Models and Statistical Analyses for Ranking 
Data (M. A. Fligner and J. S. Verducci, eds.), pp. 20-36, Springer- Verlag, New 
York, 1993. 

[16] J. P. Keener, "The Perron-Frobenius theorem and the ranking of football 
teams", SIAM Review, Vol. 35, pp. 80-93, 1993. 

[17] J. Levin and B. Nalebuff, "An introduction to vote-counting schemes", Journal 
of Economic Perspectives, Vol. 9, pp. 3-26, 1995. 

[18] G. Laffond, J.-F. Laslier, and M. Lebreton, "Condorcet choice correspondences: 
A set-theoretical comparisons", Mathematical Social Sciences, Vol. 30, pp. 23- 
35, 1995. 

[19] G. Laffond, J. Laine, and J.-F. Laslier, "Composition-consistent tournament 
solutions and social choice functions", Social Choice and Welfare, Vol. 13, 
pp. 75-93, 1996. 



23 



[20] R. B. Myerson, "Axiomatic derivation of scoring rules without the ordering 
assumption", Social Choice and Welfare, Vol. 12, pp. 59-74, 1995. 

[21] T. Wei, "The Algebraic Foundations of Ranking Theory", Ph.D. Thesis, 
Cambridge University, 1952. 

[22] M. G. Kendall, "Further contributions to the theory of paired comparisons", 
Biometrics, Vol. 11, pp. 43-62, 1955. 

[23] M. Hasse, "Uber die Behanglung graphentheoretischer Probleme unter 
Verwendung der Matrizenrechnung" , Wissenschaftliche Zeitschrift Techniscen 
Universitat Dresden, Vol. 10, pp. 1313-1316, 1961. 

[24] C. Ramanujacharyulu, "Analysis of preferential experiments", Psychometrika, 
Vol. 29, pp. 257-261, 1964. 

[25] L. Katz, "A new status index derived from sociometric analysis", 
Psychometrika, Vol. 18, pp. 39-43, 1953. 

[26] G. L. Thompson, "Lectures on game theory, Markov chains and related topics." 
Sandia Corporation Monograph SCR-11, 1958. 

[27] M. Taylor, "Graph-theoretic approaches to the theory of social choice" , Public 
Choice, Vol. 4, pp. 35-47, Spring 1968. 

[28] G. Laffond, J.-F. Laslier, and M. Lebreton, "The bipartisan set of a tournament 
game", Games and Economic Behavior, Vol. 15, pp. 182-201, 1993. 

[29] I. A. Ushakov, "The problem of choosing the preferable object", Izv. Akad. 
Nauk SSSR. Tekhn. Kibernet., No. 4, pp. 3-7, 1971. (In Russian). 

[30] I. A. Ushakov, "The problem of choosing the preferred element: An application 
to sport games", in Management Science in Sports (R. E. Machol, S. P. Ladany, 
and D. G. Morrison, eds.), pp. 153-161, North-Holland, Amsterdam, 1976. 

[31] S. T. Goddard, "Ranking in tournaments and group decisionmaking", 
Management Science, Vol. 29, pp. 1384-1392, 1983. 

[32] V. S. Levchenkov, "Entropy analysis of the majority rule", Automation and 
Remote Control, Vol. 51, pp. 517-524, 1990. 

[33] V. S. Levchenkov, "Choice without Arrow's axiom on the independence of 
irrelevant alternatives", Doklady Akademii Nauk SSSR, Vol. 322, pp. 651-655, 
1992. (in Russian). 

[34] K. A. Berman, "A graph theoretical approach to handicap ranking of 
tournaments and paired comparisons" , SIAM Journal on Algebraic and Discrete 
Methods, Vol. 1, pp. 359-361, 1980. 

[35] D. J. Daley, "Markov chains and a pecking order problem", in Interactive 
Statistics (D. McNeil, ed.), pp. 247-254, North-Holland, Amsterdam, 1979. 

[36] V. S. Levchenkov, "Choosing by tournament relations", Doklady Akademii 
Nauk, Vol. 336, pp. 324-327, 1994. (In Russian). 



24 



[37] V. S. Levchenkov and I. L. Grivko, "Some properties of self-consistent choice", 
Doklady Akademii Nauk, Vol. 327, pp. 42-47, 1992. (In Russian). 

[38] V. S. Levchenkov, "Axiomatic approach to the self-consistent choice", Doklady 
Akademii Nauk, Vol. 330, pp. 173-176, 1993. (in Russian). 

[39] I. L. Grivko and V. S. Levchenkov, "Intrinsic properties of the self-consistent 
choice rule", Automation and Remote Control, Vol. 55, pp. 689-697, 1994. 

[40] V. S. Levchenkov, "Self-consistent description of the manipulation by 
alternatives", Doklady Akademii Nauk, Vol. 331, pp. 567-570, 1993. (in 
Russian) . 

[41] V. S. Levchenkov, "Strategic scheme for group choice", Doklady Akademii Nauk, 
Vol. 341, pp. 320-323, 1995. (in Russian). 

[42] H. A. David, "Ranking from unbalanced paired-comparison data" , Biometrika, 
Vol. 74, pp. 432-436, 1987. 

[43] D. J. Cowden, "A method of evaluating contestants", American Statistician, 
Vol. 29, No. 2, pp. 82-84, 1975. 

[44] V. D. Ginovker, "On one approach to evaluating objects from paired comparison 
results." VINITI Manuscript No. 3431-81DEP, 1981. (In Russian). 

[45] J. H. Smith, "Adjusting baseball standings for strength of teams played", 
American Statistician, Vol. 10, No. 3, pp. 23-24, 1956. 

[46] H. Gulliksen, "A least squares solution for paired comparisons with incomplete 
data", Psychometrika, Vol. 21, pp. 125-134, 1956. 

[47] G. E. Noether, "Remarks about a paired comparison model", Psychometrika, 
Vol. 25, pp. 357-367, 1960. 

[48] P. S. H. Leeflang and B. M. S. Van Praag, "A procedure to estimate relative 
powers in binary contacts and an application to Dutch football league results" , 
Statistica Neerlandica, Vol. 25, pp. 63-80, 1971. 

[49] R. J. Leake, "A method for ranking teams: With an application to college 
football", in Management Science in Sports (R. E. Machol, S. P. Ladany, and 
D. G. Morrison, eds.), pp. 153-161, North-Holland, Amsterdam, 1976. 

[50] H. F. Kaiser and R. C. Serlin, "Contributing to the method of paired 
comparisons", Applied Psychological Measurement, Vol. 2, pp. 421-430, 1978. 

[51] A. I. Vereskov, "Extended Methods of Regression Analysis in the Problems of 
Identification of Nonlinear Models", Ph.D. Thesis, VNIISI, Moscow, 1986. (In 
Russian) . 

[52] V. M. Pliner, "On one approach to incomplete paired comparisons", in Abstracts 
of the Fourth All-Union Conference on Applications of Multivariate Statistical 
Analysis in Economics and Quality Evaluation, (Tartu), pp. 29-30, 1989. (In 
Russian) . 



25 



[53] P. Yu. Chebotarev, "Generalization of the row sum method for incomplete 
paired comparisons", Automation and Remote Control, Vol. 50, pp. 1103-1113, 
1989. 

[54] P. Yu. Chebotarev, "Aggregation of preferences by the generalized row sum 
method", Mathematical Social Sciences, Vol. 27, pp. 293-320, 1994. 

[55] E. Shamis, "Graph-theoretic interpretation of the generalized row sum method" , 
Mathematical Social Sciences, Vol. 27, pp. 321-333, 1994. 

[56] P. Yu. Chebotarev and E. Shamis, "On the proximity measure for graph vertices 
provided by the inverse Laplacian characteristic matrix", in Abstracts of the 
Fifth Conference of the International Linear Algebra Society, (Atlanta), pp. 30- 
31, Georgia State University, 1995. 

[57] P. Yu. Chebotarev and E. Shamis, "Characterizations of scoring methods for 
preference aggregation", Annals of Operations Research, 1998. (Forthcoming). 

[58] E. Zermelo, "Die Berechnung der Turnier-Ergebnisse als ein Maximumproblem 
der Wahrscheinlichkeitsrechnung", Math. Zeit, Vol. 29, pp. 436-460, 1928. 

[59] R. A. Bradley and M. E. Terry, "The rank analysis of incomplete block designs. 
1. The method of paired comparisons", Biometrika, Vol. 39, pp. 324-345, 1952. 

[60] M. Taylor, "Influence structures", Sociometry, Vol. 32, pp. 490-502, 1969. 

[61] N. E. Friedkin, "Theoretical foundations for centrality measures", American 
Journal on Sociology, Vol. 96, pp. 1478-1504, 1991. 



26 



